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ABSTRACT 

In this paper, we consider compressed sensing (CS) of block- sparse 
signals, i.e., sparse signals that have nonzero coefficients occurring 
in clusters. An efficient algorithm, called zero-point attracting pro- 
jection (ZAP) algorithm, is extended to the scenario of block CS. 
The block version of ZAP algorithm employs an approximate £2,0 
norm as the cost function, and finds its minimum in the solution 
space via iterations. For block sparse signals, an analysis of the sta- 
bility of the local minimums of this cost function under the pertur- 
bation of noise reveals an advantage of the proposed algorithm over 
its original non-block version in terms of reconstruction error. Fi- 
nally, numerical experiments show that the proposed algorithm out- 
performs other state of the art methods for the block sparse problem 
in various respects, especially the stability under noise. 

Index Terms — Compressed sensing, sparse recovery, block 
sparse, zero-point attracting projection. 

1. INTRODUCTION 

Compressed sensing (CS) (T|, (2) addresses the problem of retriev- 
ing sparse signals from under-determined linear measurements. It 
enjoys the advantage of reducing computational complexity in the 
measurement stage, and therefore has shown a great potential in ap- 
plications such as MRI imaging [3), wireless communication |4), 
pattern recognition (51, and source coding J6). On the part of signal 
reconstruction in CS, one of the key problems is to retrieve the spars- 
est solution, i.e., the minimum Iq norm solution to the equations of 
linear constraints: 



min||x||o s.t. y = Ax, 



(1) 



where x G W 1 is the unknown sparse signal, y £ R m is the 
measurement, and typically m < n. Unfortunately, Iq norm mini- 
mization problem is generally an NP hard problem. Previous work 
including (7) and (T) have shown that under some conditions, the 
sparsest solution can be obtained via convex relaxation, such as 
Basis Pursuit (BP). Another popular method for CS recovery prob- 
lem is based on greedy pursuits, and its representative is orthogonal 
matching pursuit (OMP) (§). 

The block spare problem for compressed sensing was first intro- 
duced by Eldar et.al in J9]. The authors have shown that sampling 
problems over unions of subspaces can be converted into block- 
sparse recovery problems. Examples in applications can be found 
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in 1101 . 1111 and 1121 . Mathematically, a block-sparse signal can be 
represented as follows: 



Xl\ 



T 
X 2 , 



T 



(2) 



where x; = [x(iD — D + 1), • ■ ■ , x(iD)] is the zfh block of x with 
length D, and n — N x D. A signal is K block-sparse if at most K 
out of the TV blocks of the signal are non-zero. Similar to (Q3, the ^2,0 
norm minimization for the block-sparse problem can be formulated 
as: 

min||x|| 2 ,o, s.t. y = Ax, (3) 



Ed 



Xfe 



(4) 



From © it is clear that the l 2 ,o norm can be inteipreted as the num- 
ber of non-zero blocks of the signal. Like the Iq minimization prob- 
lem ([TJ, solving 10 is also NP-hard. Although all the conventional 
recovery algorithms in CS is also applicable to the block CS prob- 
lem, these algorithms cannot take advantage of the essential block- 
sparse characteristic of signals. To remedy this, Eldar et.al intro- 
duced two algorithms in 1131 and J9): the /2,1-opt and the Block or- 
thogonal matching pursuits (BOMP). However, like their ancestors, 
these algorithms have their inherit drawbacks: /2,1-opt is quite slow 
and becomes worse as dimension increases; BOMP is faster, but its 
estimation accuracy is poorer in the presence of noise perturbation. 

In contrast, a recently proposed algorithm called zero-point at- 
tracting projection (ZAP) II 141 is an efficient sparse reconstruction 
method based on an idea different from the aforementioned convex 
relaxations and greedy pursuits: The authors choose a smooth func- 
tion to approximate the Iq norm and then finds its minimum in the 
solution space via iterations. Their simulations show that ZAP re- 
quires fewer measurements for exact reconstruction than the refer- 
enced algorithms in the experiment settings, while having tractable 
computational complexity. It is then interesting to explore the block- 
sparse reconstruction methods based on the idea of ZAP. 

In this paper, the ZAP algorithm is extended to the block sparse 
model. The block ZAP algorithm (BZAP) employs a smoother cost 
function to approximate the h.o norm of the block sparse input, and 
then minimize this function via iterations. An analysis of the stabil- 
ity of the local minimums of the cost function gives a lower bound 
on the reconstruction error for BZAP than the original ZAP, approx- 
imately by a factor of \j\fD. Simulations show that BZAP outper- 
forms other state-of-the-art methods for the block sparse problem 
(BOMP, Z2,i-opt) both in terms of incidence of exact recovery in 



noiseless case, and the mean square deviation in the case of noise- 
contaminated measurements. 

The remainder of the paper is organized as follows: section 2 
presents the formulation of the BZAP algorithm. In section 3, an 
analysis of the h stability of the local minimum of the cost function 
for BZAP is offered. Section 4 presents simulation results compar- 
ing BZAP with BOMP, Z 2 ,i-opt and the original ZAP. Finally, the 
whole paper is concluded in section 5. 

Notation. Throughout this paper, we denote vectors by boldface 
lowercase letters, and matrices by boldface upper case letters. Given 
a matrix A, A* is its Hermitian conjugate. A^ denotes the pseudo 
inverse of A, that is, if A has full row rank or full column rank, then 



,t 



A*(AA*)-\ 



A has full row rank; 
A has full column rank. 



(5) 



Block support T is a subset of { 1 . . . iV} indicating the non-zero 
blocks of x, and T c is its complement. We use xt to denote the 
vector formed by the blocks in x indexed by T, and At the sub- 
matrix that lies in the column blocks indexed by T. Notation || ■ | 
takes either the Euclid norm of a vector or the I2 operator norm of a 
matrix. 

2. BLOCK ZAP ALGORITHM 

This section aims to extend the ZAP algorithm to the block sparse 
problem. One chief idea of BZAP is to employ a 'smoother' func- 
tion: 



J(x) = ^F(l|x & l 



(6) 



to approximate the £2,0 norm of x. Of course, there is a great liberty 
in the choice of the function F in (|6). But to reduce computation 
complexity, we select 



?„(«,) = / 2aM-aV \w\<± 
\ 1 elsewhei 



(7) 



in the implementations, since its derivative is linear. Now from I0 
we see that the £2,0 norm of x can be approximated as: 



|x|ko«5>«(||x h ||), 



So the problem ((3} is transferred to 



mm^F Q (||x fc ||), s.t. y = Ax. 



(8) 



(9) 



Traditional methods of steepest descent together with a 'projection' 
step can be used to solve {5). That is, in the ith iteration, the solution 
is updated along the negative gradient direction of the sparse penalty, 
which in effect attracts the solution to the zero point: 



x(i + 1) = x(t) - k • VJ(x(t)). 



(10) 



Since x(£ + 1) is generally not in the solution space, the next step is 
to project it back to the hyperplane of Ax = y: 



x(i + l) = Px(t+l) + Q, 



(11) 



where P = I — A^ A is named as projection matrix and Q = A^y. 
The attraction step dlOt and projection step i ll It are used alternately 



Table 1. Procedure Outline of BZAP 



Input: a, k, A, y; 

Initialize BZAP: x (0) = A f y, t = 0. 

Repeat: (for time instant i); 

Update x(t + 1) with the zero attraction by l |IO> and ([8}; 

Project x(t + 1) back to the solution space by Jilt : 

Update the index: t — t + 1. 
Until: Block ZAP stop criterion is satisfied. 



in the iterations, hence the name of zero-point attracting projection. 
The procedure of BZAP is summarized in TABLEQ] 

Finally, we remark on the choice of parameters for the BZAP 
algorithm: 

The choice of a: According to ((7}, parameter a determines the 
range of effect of the cost function. There is a tradeoff in the choice 
of a since a small a leads to a bad approximation of the I2.0 norm, 
and produces many local minimums, while an overly large a limits 
the effective range. Empirically we have found that BZAP performs 
the best when 1/q is around the square root of the variation of the 
non-zero entries in x. 

The choice of k: The step length k determines the speed of con- 
vergence and the accuracy of the estimation. A large k will result in 
a fast convergence but a poor estimation. In our simulations, k is de- 
creased as the iterations approaches convergence, in order to ensure 
both speed of convergence and accuracy. More specifically, we let 
k decrease by a factor of r\ (r\ < 1) whenever the cost function (|6j 
starts to increase. 

Stop conditions: The iteration dlOt and i ll It is terminated when 
any of the two following conditions is satisfied: (a) The total number 
of reductions of step length k reaches a predefined number C\ , or (b) 
The total number of iterations exceeds a predefined number C'2 ■ 

3. STABILITY OF THE LOCAL MINIMUM POINT 

In this section, we consider the problem of estimating x from the 
following noisy measurements: 



y = Ax + v. 



(12) 



While in 1151 , the authors have discussed the convergence of the 
ZAP iterations, in this work we mainly consider the stability of local 
minimums of the cost function of BZAP under noise perturbation. 
Suppose a satisfies 

l/a< ||X*||, k£T, (13) 

define the closed ball B(x, d) as a neighborhood of x, where 

d= min(l/a, ||x fc || - 1/q). 

keT 

Let L be the solution space: 

i:=jx£l":y = Ax}, 

where y is the measurement given in l !12t . Then, regarding the sta- 
bility of the local minimizer of (|9j in the noise-contaminated mea- 
surements, we have the following theorem: 

Theorem 1. Suppose x is a block sparse signal and At has full 

column rank, then the minimizer x of function {6j in the region 
L P| -B(x, d) satisfies 



||x - x|| < 2s/N{\ + ||a£At= 11)11 A f v|| + ||A T v|| 



(14) 



Proof. Let 

<Jx = x — x (15) 

be the difference between the local minimum of the cost function 
and the real signal. The aim is to bound ||<Sx|| with v. Obviously, 

||5x|| < ||<5xt|| + ||<5xtc||. (16) 

Then we will derive bounds on ||<5xt| and ||<5xt<: || respectively: 
First, consider the bound on ||(5xt c ||. If x £ B(x, d), then 

||x fc -x fc ||<d, k = l...N, 

and it follows from the definition of d that 

||x fc ||>l/a, keT; 

||x fe ||<l/Q, t£f. (17) 

Therefore, with the cost functions defined in ((6), (|7j, we have 

a||xHki < Yl F «(ll x fclD ^ 2a||xTc|| 2 ,i- (18) 

fcST<= 

Next, we will prove 

[|<5xtHI < 2V r ZV]|A t v|| (19) 

by differentiating between the following two situations: 

1) If || A f v|| > d, then {[9} automatically holds. 

2) If |j A t vj| < d, we have 



||&X T c|| < ||<5x T c|| 2 ,i 
rv -^ — ^ 



(20) 
(21) 



fc£T c 



= min- V F Q (||x fc ||),s.t. ||x|| < d, Ax = v (22) 
x a *■ — ' 

keT" 

< min2||x|| 2 ,i,s.t.||x|| < d, Ax = v (23) 

X 

< 2||A + v|| 2 ,i (24) 

< 2\/7V||A t v||, (25) 

where the definition of <5x is used in the derivation of d22t , relation 
J18t in the derivation of d23t , and the fact that A^ v is a feasible point 
for the constraint of ( 123) in the derivation of ( 1241 . To conclude, d!9l l 
holds in both situations. 

Finally, we derive abound on ||5xt||. Since 

At5xt = v — At c 5xt<=, 
we have 

<5xt = Ay(v — At c <5xt<=), 
therefore by triangular inequality and (119) , 

\\6xt\\ < ||At,v|| + ||At,ATc(JxTc|| 

< || At,v|| + 2Vn\\aI,Atc II ||A + v|| 2 . (26) 

Then, combining d25l l and l |26l l yields the final result d!4) . D 

Now, we remark on the improvement of BZAP over ZAP: Since 
ZAP can be seen as the D = 1 special case of BZAP, when theorem 
1 is applied to ZAP, the bound becomes 

||x-x|| <2^(l + ||At,A T c||)||A t v|| + ||At,v||. (27) 

It will be shown later that the term ||Ayv|| in d!4t is equivalent 
with the error of a so-called 'oracle estimator', which gives a lower 
bound on the mean square error for the recovery problem. The other 
term, 2^(1+ ||A^Atc ||)|| A f v|| in <[27j is reduced by BZAP by a 
factor of VD in d!4t . This reveals that the reconstruction via BZAP 
is more stable than via ZAP in the case of noisy measurements. 



4. SIMULATION RESULTS 

In this section, the proposed BZAP algorithm is compared with the 
conventional ZAP, BOMP, and Z2,i-opt algorithm. In all examples, 
the measurement matrix A has 40 rows and 100 columns, with inde- 
pendent entries following the distribution of A/"(0, 1). The block is 
of the size D = 4. The locations of nonzero blocks in the unknown 
sparse signal x are randomly chosen, and the values of nonzero ele- 
ments are independently drawn from the Rademacher distribution. 

For the BZAP algorithm, we set k = 1, a = 1, 57 = 0.1, 
G\ = 4, and C2 = 1200; Therefore it's easily checked that con- 
dition d!3l > in theorem 1 is always satisfied. For ZAP and £2,1 -opt, 
we adopt the same stop conditions and control of step size as in the 
implementation of BZAP. 

4.1. Recovery rate for different block sparsity 

In this first experiment, the exact recovery rate for different algo- 
rithms in the noiseless case is compared. We define exact recovery 
when the squared deviation||x — x|| 2 /||x|| 2 is smaller than 10~ 6 . 
One thousand independent simulations are conducted to calculate 
the empirical exact reconstruction rate. 

As is shown in Fig.l, the proposed BZAP algorithm outperforms 
all the other referenced algorithms in the experiment condition. That 
is, BZAP can achieve exact reconstruction of sparse signals when 
there are more non-zero elements: while other algorithms exactly 
reconstruct the signal when K is no more than 3, BZAP can achieve 
this when K is up to 4. ZAP gives a poor estimation because it is 
the only one of the algorithms that doest not employ the block sparse 
nature of the signal. 

4.2. Mean square deviation (MSD) in the presence of noise 

In this experiment, the noise-contaminated measurements is formu- 
lated as in d!2t . The observational signal-to-noise ratio (SNR) is 
defined as 

/IIAxll 2 \ 
SNR = lOlg \-4r . (28) 



In the simulation the SNR ranges from lOdB to 50dB. The noise 
vector v is first generated with independent entries following the 
normal distribution and then re-scaled to the fit the designed SNR. 

To compare the reconstruction error, the mean-square deviation 
associated with different algorithms is calculated, which is defined 
as follows: 



MSD 



E||x- 



(29) 



E||x|| 2 ' 

To calculate the empirical expectation in d29) , we take the average 
of the squared norms over 10 5 independent simulations. 

Regarding the MSD lower bound, consider the following ora- 
cle estimator: suppose the support T is known, then the minimum 
variance unbiased estimate of x is the least square estimate: 



A^y. 



(30) 



The reconstruction error is || A^ v||, therefore the MSE is given by 



E(||x-x|| 2 ) = ctVKAtA-z 



(31) 



which should be lower than the achievable MSE for any practical 
estimators. 

The simulation results are shown in Fig.2. In this experiment, 
the proposed BZAP algorithms again outperforms other estimators 
in terms of MSD, and in fact closely follows the oracle bound. The 



BOMP algorithm, although guarantees higher exact recovery rate in 
the noiseless case, is very unstable under the perturbation of noise. 



5. CONCLUSION 

In this paper, we have extended the ZAP algorithm to the block- 
sparse problem, by introducing a cost function to approximate the 
£2,0 norm of the signal. The stability of the local minimum of the cost 
function in BZAP is studied, which reveals an advantage of BZAP 
over the original ZAP by employing block sparsity of block sparse 
signals. Finally, simulation results show that BZAP out-performs 
BOMP, /2,1-opt and the original ZAP both in terms of the incidence 
of exact recovery in the noiseless case, and the mean square devia- 
tion in the noisy measurements. 



Empirical probability of exact reconstruction 




Fig. 1. Recovery of an input signal from y = Ax, where x is a 
block sparse signal with a block sparsity level of K. 



Empirical mean square deviation 
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Fig. 2. Recovery of an input signal from y = Ax + v, where x is a 
block sparse signal with a block sparsity level of K — 4. 



